1. Field of the Invention
The present invention relates to a semiconductor neural circuit device and an operating method thereof, and more particularly, it relates to the structure of a connection matrix of low power consumption and a small area, which is suitable for high integration.
2. Description of the Background Art
In recent years, various types of parallel arithmetic processing techniques have been proposed on the model of vital cells. Such a parallel arithmetic processing technique employs a model called a neural network. A neuron model employed in such a neural network is now described.
FIG. 1 shows a neuron unit which is provided in a neural network. Referring to FIG. 1, a unit i corresponding to a neuron includes signal an input part A, a conversion part B for converting signals supplied from the input part A along a prescribed rule, and an output part C for outputting the data converted in the conversion part B. The input part A has a prescribed weight (synapse load) W.sub.ij in correspondence to each unit. For example, a signal S.sub.k received from a unit k is converted into a signal Sk.multidot.W.sub.ij with multiplication of a weight W.sub.k, and then transferred to the conversion part B.
The conversion part B passes through a prescribed function the summation ##EQU1## of the signals transferred through the input part A, and outputs an output signal S.sub.i.
A nonlinear monotonously increasing function, such as a sigmoid function shown in FIG. 2, is utilized as a function g(u) for converting the input data in the conversion part B. The sigmoid function is expressed as: ##EQU2## where u.sub.0 represents a predetermined threshold value. Such a neuron model is employed in the parallel arithmetic processing technique of a neural network model called Hopfield model. This neural network model is generally adopted to solve various problems such as an optimization problem through simulation with a serial processing computer.
However, it is inefficient to simulate such a neural network having parallel processability in essence with a computer which is essentially a serial processing unit. Therefore, such a neural network has been implemented as an electronic circuit.
FIG. 3 shows exemplary structure of a conventional neural network which is implemented as an electronic circuit. The neural network shown in FIG. 3 is disclosed in U.S. Pat. No. 4,660,166 to Hopfield, for example. Referring to FIG. 3, the conventional neural network includes amplifiers A.sub.i, A.sub.i, A.sub.j, A.sub.k and A.sub.k serving as neuron units, data input lines I.sub.i, I.sub.j and I.sub.k, and data output signal lines X.sub.i, X.sub.i, X.sub.j, X.sub.j, X.sub.k and X.sub.k.
The data input lines I.sub.i to I.sub.k correspond to dendrites, and the data output lines X.sub.i and X.sub.i to X.sub.k and X.sub.k correspond to axons. Resistive elements having conductance T.sub.ij are provided on crossings of the input lines I.sub.i to I.sub.k and the data output lines X.sub.i and X.sub.i to X.sub.k and X.sub.k. The input line I.sub.i is coupled with the output line X.sub.j through this resistive element T.sub.ij. The amplifiers A.sub.i and A.sub.i output signals which are complementary to each other. Thus, the output signal lines can be paired as complementary signal line pairs, to implement both excitatory connection and inhibitory connection. In the case of excitatory connection, the data input line I.sub.i is coupled with the data output line X.sub.j through the conductance T.sub.ij, as shown in FIG. 4A. In the case of inhibitory connection, on the other hand, the data input line I.sub.i is coupled with the complementary data output line X.sub. j through the conductance T.sub.ij as shown in FIG. 4B. Signal input and output characteristics of the amplifiers A.sub.i to A.sub.k are expressed by the sigmoid function shown in FIG. 2. The operation of this neural network is now briefly described.
In this model, each neuron unit (amplifier) can be connected with any unit. It is assumed here that u.sub.i represents a potential appearing at an input terminal of the amplifier A.sub.i, and V.sub.i represents a potential appearing at its output terminal. As hereinabove described, there is the following relation in this case: EQU V.sub.i =g(u.sub.i)
As described in the aforementioned Hopfield patent in detail, the following equation holds in one neuron unit. ##EQU3##
R.sub.i : input resistance of amplifier A.sub.i
C.sub.i : input capacitance of amplifier A.sub.i
I.sub.i : current flowing in input signal line I.sub.i
When values are given to the aforementioned parameters T.sub.ij, I.sub.i, g(u.sub.i), C.sub.i and R.sub.i, it is possible to simulate the time-dependent change in state of respective units provided in this neural network through the aforementioned nonlinear differential equation. However, when the states of the respective neurons are one by one changed in time, programs will be impractically excessive as the number of the units forming the neural network is increased. Therefore, Hopfield introduces the following energy function: ##EQU4## as a quantity expressing the property of the overall neural network. This energy function is in the same form as spin Hamiltonian, which is used in an Ising model in material theory. The Ising model is employed for illustrating a phase transition phenomenon of a ferromagnetic material in statistical mechanics, and gives energy of the overall system when spins exhibiting plus and/or minus states interact with each other to cause state transition. It is known that this energy function takes the minimum value when the system is in an equilibrium state.
In the neural network, therefore, the goal is found in a potential V.sub.i which minimizes the energy function E. Namely, the electronic circuit shown in FIG. 3 has such function that the respective amplifiers A.sub.i to A.sub.k operate in parallel with each other, to output an output signal V.sub.i which minimizes the energy function E with respect to supplied data.
When the curve of the function showing the input and output characteristics of the aforementioned amplifiers is abrupt, the form of the energy function E is simplified. In this case, the output of each neuron unit is substantially zero or close to the maximum output of 1, if the system of the neural network is in a stable state having low energy. For this case, the energy function E is given as follows: ##EQU5##
In the network according to the Hopfield model, obtained are such output data that the energy E of the network is settled at the minimum value. Therefore, a resistive matrix formed by the coupling elements stores certain patterns or data in accordance with program states of the resistive coupling elements T.sub.ij, and can decide match/mismatch between input data and the stored pattern or data. Thus, such a neural network can be applied to an associative memory circuit or a pattern discriminator.
FIG. 5 shows the structure of such a programmable resistive coupling element. The structure of the programmable coupling element shown in FIG. 5 is disclosed in IEEE Computer, March 1988, pp. 41 to 49.
Referring to FIG. 5, the conventional programmable coupling element includes switching elements S.sub.1, S.sub.2, S.sub.3 and S.sub.4, resistive elements R.sup.+ and R.sup.-, and random access memory cells RM.sub.1 and RM.sub.2. The switching elements S.sub.1 and S.sub.4 enter on states in response to the signal potential on a signal line X.sub.j. The switching element S.sub.2 enters an on state in response to information stored in the random access memory cell RM.sub.1. The switching element S.sub.3 enters an on state in response to information stored in the random access memory cell RM.sub.2. The resistive element R.sup.+ is connected to a source potential V.sub.CC, and the resistive element R.sup.- is connected to a ground potential V.sub.SS. These resistive elements R.sup.+ and R.sup.-, which are of high resistance values, have current limiting function.
When the random access memory cell RM.sub.1 stores data "1" and the random access memory cell RM.sub.2 stores data "0", the switching element S.sub.2 enters an on state and the switching element S.sub.3 enters an off state. Therefore, current is flown from the resistive element R.sup.+ into an input terminal of an amplifier A.sub.i in response to the signal potential on the signal line X.sub.j, thereby to express a positive coupling degree T.sub.ij.
When data "0" is written in the random access memory cell RM.sub.1 and data "1" is written in the random access memory cell RM.sub.2, on the other hand, the switching element S.sub.2 enters an off state and the switching element S.sub.3 enters an on state. In this case, current is flown from the input terminal of the amplifier A.sub.i to the ground potential V.sub.SS in response to the signal potential on the signal line X.sub.j, thereby to express a negative coupling degree T.sub.ij.
A coupling degree 0 is expressed by writing data "0" in both of the random access memory cells RM.sub.1 and RM.sub.2 and bringing the switching elements S.sub.2 and S.sub.3 into off states
Such random access memory cells RM.sub.1 and RM.sub.2 are generally formed by static random access memory cells, which require no refresh function.
A perceptron proposed by F. Rosenblatt is known as a neural network model. FIG. 6 shows simplified structure of the perceptron.
Referring to FIG. 6, four nerve cells 501 to 504 are connected to a nerve cell 105 through synapse loads W.sub.1, W.sub.2, W.sub.3 and W.sub.4, respectively. Assuming that X.sub.1 to X.sub.4 represent quantities of stimuli outputted from the respective nerve cells 501 to 504, the summation of the quantities of stimuli received in the nerve cell 505 is expressed as ##EQU6## The nerve cell 505 enters a firing state and its output goes to "1" when the summation .SIGMA.W.sub.i X.sub.i is higher than a predetermined threshold value h, while the output goes to "0" when the former is lower than the latter.
This perceptron can be so set that an output Y.sub.1 of the nerve cell 505 goes high only when inputs X.sub.1 to X.sub.4 have certain patterns by appropriately programming the degree of coupling of the synapse loads W.sub.1, W.sub.2, W.sub.3 and W.sub.4. Thus, this perceptron can serve as a discriminator for input patterns (X.sub.1, X.sub.2, X.sub.3, X.sub.4).
A plurality of such perceptrons are connected in a multistage manner as shown in FIG. 7, thereby to increase ability for pattern discrimination.
The electronic circuit of the neural network has the aforementioned structure, and current stationarily flows during the operation through resistors of the coupling elements providing the degree of coupling between signal lines. In the coupling element shown in FIG. 5, for example, an output signal line of an amplifier A.sub.j merely drives the switching elements S.sub.1 and S.sub.4 and it is not necessary to drive the input signal line of the amplifier A.sub.i, whereby an output load of the amplifier A.sub.j is reduced. In this case, however, current flows between the input terminal of the amplifier A.sub.i and the source potential V.sub.CC or the ground potential V.sub.SS. Thus, standby-state current normally flows during the operation, to increase power consumption.
Although various structures have been proposed to couple signal lines simply through resistive elements, current flows between the signal lines also in this case, to increase power consumption.